Scoring metric for physical activity performance and tracking

ABSTRACT

Certain aspects of the present disclosure provide a method for assessing the performance of a physical activity, including: recording motion capture data while a training subject demonstrates a physical activity sequence; identifying one or more primary body elements based on the motion capture data; deriving one or more path characteristic metrics based on a state variable set and the one or more primary body elements, wherein the state variable set defines the state of a body at any given time; and defining an ideal activity path of the physical activity sequence based on the one or more path characteristic metrics.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to pending U.S. patent application Ser.No. 16/880,693, filed May 21, 2020, which claims the benefit of U.S.Provisional Patent Application No. 62/851,009, filed on May 21, 2019,the entire contents of each application is incorporated herein byreference.

INTRODUCTION

Aspects of the present disclosure relate to systems and methods forquantifying and monitoring physical activities based on motion data, andin particular, to quantitatively assessing the performance of a physicalactivity given subject motion capture data provided by a motion capturesystem.

In a physical rehabilitation setting, patients are often prescribedphysical therapies, which may include specific, physical activities,such as exercises targeting specific movements of specific limbs andjoints. Typically, a patient is given written instructions for when andhow to perform the physical activities (e.g., a certain number ofrepetitions of a specific exercise every twelve hours). Historically,patients have also needed to go to a physical therapy clinic, or to havea clinician visit them at home, to monitor their physical therapy and toget feedback and coaching to maximize compliance with their physicaltherapy. Such on-site monitoring generally improves the efficacy of thephysical therapy, for example, by ensuring that it is performedcorrectly and consistently. However, this conventional practice istime-consuming, expensive, and logistically challenging. Moreoverin-clinic or in-home physical therapy coaching may not be available tothose with limited mobility or means. While patients can performprescribed physical therapies on their own without professional support,there is no guarantee that the patients will follow instructions and useproper form—which is critical to the efficacy of the prescribed physicaltherapies. In fact, unsupported physical therapy frequently leads toinferior patient outcomes, higher chances of re-injury, and the like.

Notably, the same issues faced in the physical therapy context arepresent in other contexts, such as physical fitness training forperformance improvement rather than injury recovery, in coaching ofathletes for various sports, and in any other context where theconsistency and quality of body motions may improve a desired outcome.

Accordingly, what are needed are systems and methods for quantitativelyassessing the performance of a physical activity based on motion capturedata.

BRIEF SUMMARY

Certain aspects provide a method for assessing the performance of aphysical activity, including: recording motion capture data while atraining subject demonstrates a physical activity sequence; identifyingone or more primary body elements based on the motion capture data;deriving one or more path characteristic metrics based on a statevariable set and the one or more primary body elements, wherein thestate variable set defines the state of a body at any given time; anddefining an ideal activity path of the physical activity sequence basedon the one or more path characteristic metrics.

Other aspects provide processing systems configured to perform theaforementioned methods as well as those described herein;non-transitory, computer-readable media comprising instructions that,when executed by one or more processors of a processing system, causethe processing system to perform the aforementioned methods as well asthose described herein; a computer program product embodied on acomputer readable storage medium comprising code for performing theaforementioned methods as well as those further described herein; and aprocessing system comprising means for performing the aforementionedmethods as well as those further described herein.

The following description and the related drawings set forth in detailcertain illustrative features of one or more embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

The appended figures depict certain aspects of the one or moreembodiments and are therefore not to be considered limiting of the scopeof this disclosure.

FIG. 1 depicts an example relationship diagram depicting various aspectsthat are described herein.

FIG. 2 depicts an example process for developing and implementing a pathtracking model.

FIG. 3 depicts an example of skeletonization output from a motioncapture source with primary body elements identified.

FIG. 4 depicts an example of using primary body element unit vectors toderive path characteristic metrics to measure a subject's physicalactivity path.

FIG. 5 depicts an example of activity path normalization.

FIG. 6 depicts an example method of utilizing averaging to combineseveral normalized activity path samples into one mean normalized path.

FIG. 7 depicts an example method of implementing the mean pathcompletion time to convert the mean normalized path into an idealactivity path expressed as a function of time.

FIG. 8 depicts an example method for finding the best matching statealong an ideal activity path based on a current orientation of a primarybody element.

FIG. 9 depicts an example graphical user interface of a live activitypath tracking application.

FIGS. 10A and 10B depict examples of augmented vector-defined activitypaths.

FIG. 11 depicts an example method of assessing the performance of aphysical activity.

FIG. 12 depicts an example processing system configured to generate anduse activity path tracking models.

To facilitate understanding, identical reference numerals have beenused, where possible, to designate identical elements that are common tothe drawings. It is contemplated that elements and features of oneembodiment may be beneficially incorporated in other embodiments withoutfurther recitation.

DETAILED DESCRIPTION

Aspects of the present disclosure provide apparatuses, methods,processing systems, and computer readable mediums for quantitativelyassessing the performance of a physical activity based on motion capturedata.

Overview of Physical Activity Tracking

FIG. 1 depicts an example diagram depicting various aspects of aphysical activity tracking and assessment system, as described herein.

Generally, a physical activity (or activity sequence) is a bodilyactivity that includes a temporal sequence of states (e.g., 102 or 126)of a subject's body. Physical activities may take many different forms.For example, a physical activity may comprise an exercise or otherprescribed motion performed or practiced in order to develop, improve,or display a physical capability or skill.

A state (or activity state) 102 associated with a physical activity maygenerally include a particular position, pose, or bearing of a subject'sbody, whether characteristic or assumed for a special purpose. One ormore states in a sequence of states defining a physical activity may beconsidered key states, and key states may be used to define amathematical representation of a physical activity.

For example, the physical activity of sitting down may have a first keystate of standing and a second key state of sitting. The states inbetween standing and sitting may not be considered “key” because theymay not be important to defining the overall physical activity ofsitting. However, as described in further detail below, a path betweenstates (e.g., 106), including between key states, may be used by a pathtracking model to quantitatively assess the performance of a physicalactivity defined by its key states.

Key states may be determined by experts, such as trainers, clinicians,doctors, or the like, or determined numerically based on analyses oftemporal sequences of motion data associated with a physical activity.For example, points of inflection in a motion path of a particular bodysegment or joint may indicate a key state of a particular physicalactivity.

In some cases, a physical activity may be further defined by temporalspecifications, such moving from one state to another state within aspecified time, or holding one state for a specified time, to name justa few examples.

Each state in a physical activity state sequence may be defined withreference to individual segments or portions of a subject's body, suchas the subject's head, neck, torso, arms, hands, fingers, legs, feet,bones, and others. States within physical activity sequences may befurther defined by joints, which are generally connection points betweentwo adjoining body segments that allow for some articulation of onesegment in relation to another connected segment. In some cases, theindividual segments and joints associated with a subject may be combinedto form a mathematical body representation, such as a skeletonrepresentation, or other more featured representations, such as anavatar.

A state (e.g., 102) may be programmatically described and tracked usingone or more state variables 116 (e.g., in a set of state variables 104),which may include numeric and/or non-numeric variables. For example, aset of state variables may describe the position and orientation oflimbs and joints relative to the body or an independent frame ofreference, such as a coordinate system. In some cases, a set of statevariables (e.g., 116) may fully define the state of a subject's body.

In addition to defining states, paths between states (e.g., motion pathsor state transition paths) may also be defined. Generally a path betweenstates (e.g., 106) may be expressed as an ordered sequence of statevariable sets, where each state variable set may be derived from a fullstate variable set (e.g., 104), used for tracking transitions betweenand including any two discrete states. For example, a path may include aseries of states (e.g., poses), typically ordered by a time or sequencein which they are performed. Because each state may be described by astate variable set (e.g., 104 in FIG. 1), a path may be described as asequence (or series) of these state variable sets. In some embodiments,the sequence may be ordered using temporal values or the order may bespecified otherwise. A path between states may also be expressed as amathematical function.

Generally, an activity path between states may be multi-dimensional,such as an n-dimensional path between discrete states. Thedimensionality may depend on, for example, the number of body primaryelements associated with the activity and the reference frame in whichthey are expressed. In such cases, each dimension of the path betweenstates may be considered a path component. In some cases, a path betweenstates may be projected to a lower dimensionality, such as athree-dimensional path projected to a two-dimensional plane.

For example, an activity path based on a single joint (a type of bodyelement), may include an ordered series (e.g., in time) of three spatialdimension values for that primary body element as it moves through theactivity path, and thus be three-dimensional. As another example, anactivity path based on a single limb (another type of body element) mayinclude an ordered series of six spatial dimension values, where threespatial dimension values are associated with a first end of the limb(e.g., at a first joint) and the other three spatial dimension valuesare associated with a second end of a limb (e.g., at a second joint). Asyet another example, a body element may be represented by three spatialdimensions at a first joint location and a number of orientationreferences (e.g., in Euler angles). As a further example, an activitypath based on a plurality of primary body elements may include anordered series of spatial dimension values for each primary body elementin the plurality as it moves through the activity path. Notably, anactivity path may be based on any number of body elements and thus anynumber of spatial, orientation, and other non-spatial dimensions, suchas temporal dimensions, and others.

Physical activity state sequences may be captured and digitized througha process of motion capture, motion monitoring, motion tracking, or thelike, which all generally refer to a process for generating dataregarding a subject's kinematic motion and static poses using a varietyof electronic sensors. Generally, a motion tracking system may includehardware and software components configured to monitor a subject'sphysical activity (e.g., movements, exercises, etc.). In someembodiments, a motion capture device may include optical camera systemswith image processing, marker based tracking systems with various marker(active, passive, semi-passive, modulated) and detector (optical, radiofrequency) types, depth camera systems with object recognitionalgorithms, inertial measurement units, mechanical exoskeleton motioncapture systems, or magnetic flux measurement systems, to name a few.One example of a motion tracking system is the KINECT® sensor and itsassociated pose detection software by MICROSOFT®.

States in a physical activity state sequence may be compared to generatestate differentiation variables. For example, a plurality of candidateinter-state differentiation variables may be defined based on bodysegments and joints in order to identify or improve identification ofdifferences between states, including key states, of a physicalactivity. In some embodiments, the plurality of candidate inter-statedifferentiation variables may be tested to determine a subset ofinter-state differentiation variables that are most effective foridentifying particular states, such as key states, in captured motiondata. The selected subset inter-state differentiation variables may bereferred to as state characteristic metrics (e.g., 118), which aregenerally used by a physical activity definition model (e.g., 120) foridentification and tracking of key states of a physical activity inmotion data.

An ideal physical activity sequence may be defined, for example, bycapturing motion data of a professional performing a physical activityin a prescribed manner and thereafter defining key states of thephysical activity based on the captured motion data. As above, the keystates may be defined manually, such as by a professional, orautomatically, by analyzing the change in variables in captured motiondata during performance of the ideal physical activity sequence.Further, in some implementations, key states can be numerically definedwithout a sample of collected motion capture data, such as by use of askeletal data model.

Once a physical activity has been defined (e.g., by way of quantifiedstate characteristic metrics), a physical activity definition model(e.g., 120) may be generated to determine (e.g., recognize or identify)states, such as key states, of the physical activity in captured motiondata. Further, the physical activity definition model may compare thedetermined states with states of an ideal physical activity statesequence to “score” a subject's performance of the physical activityand/or to provide live feedback to the subject on the quality of theperformance of the physical activity. This enables, in effect, livemonitoring and feedback to a subject without the need for an on-siteprofessional.

In some embodiments, a physical activity definition model (or physicalactivity model) (e.g., 120) comprises one or more classifiers (e.g.,120A) that are configured to determine probabilities (e.g., 122) that aparticular state is represented in captured motion data. Further, foreach classifier of the physical activity model, a classifier confidencemay be determined as a quantitative assessment corresponding to theclassifier's performance in determining a correct state, classification,or category of captured motion data. Determination of particular statesvia the physical activity definition model may further lead todetermination that a defined physical activity, which comprises some orall of the determined states, is represented in the captured motiondata.

The combination of classifiers (e.g., 120A) and state characteristicmetrics (e.g., 120B) enables a single physical activity definition modelto generate predictions regarding a plurality of defined states andphysical activities in captured motion data.

In addition to determining states with an activity definition model(e.g., 120), the performance of a physical activity may further byquantitatively assessed based on paths of body segments and jointsbetween states in the physical activity.

In some cases, the motion paths of body segments may be based onselected primary body elements (e.g., 114), which are generally elementsof a subject's body usable for defining, tracking, and assessingactivity form and/or performance. FIG. 3, described in more detailbelow, depicts an example of primary body elements in transition betweentwo states in a physical activity sequence.

Primary body elements may be representative of the path between states.For example, the transition from a first state to a second state mayinvolve significant motion of a subject's legs relative to its torso,but not its head and neck relative to its torso. Thus, the legs may beconsidered primary body elements in such a scenario. In someembodiments, a primary body element may be associated with one or morestate variables (e.g., 116).

In some cases, primary body elements may be identified directly by anexpert practitioner. Beneficially, during path tracking modelformulation, the expert may refine a list of path characteristic metricsassociated with primary body elements in order to place more emphasis ontracking body elements of greater interest.

In other embodiments, the primary body elements may be determinedalgorithmically. For example, primary body elements may be determinednumerically from analysis of state variables (e.g., 104) between states(e.g., key states) in a physical activity sequence. For example, anumerical approach may be based on the mechanical work performed by eachbody element as a subject moves through an activity sequence. Analyzingthe quantified work performed may inform identification of primary bodyelements for that physical activity sequence, or portion thereof.

A path characteristic metric (e.g., 124) is generally a state variableassociated with or derived from one or more primary body elements usedto define significant components of an activity sequence.

Path characteristic metrics may be used to define activity paths (e.g.,108). For example, an activity path or trajectory (e.g., 108) maygenerally be represented by one or more path characteristic metrics(e.g., a set of path characteristic metrics) for each intermediate state(transition region) between and including an activity path's beginningkey state and ending key state.

An ideal activity path (e.g., 110B) is generally a desired activity path(e.g., as defined by an expert) associated with the full or partialcourse of a physical activity sequence.

A path similarity measure (e.g., 110C) is generally a numeric valuewhich quantifies the similarity between actual activity paths (ortrajectories) and an ideal activity path (e.g., 110B).

A path tracking model (e.g., 110) may generally include one or more pathsimilarity measures (e.g., 110C) used to assess the complete motion in aphysical activity sequence as compared to an ideal activity path (e.g.,110B). The path tracking model may generate path tracking scores (e.g.,112), which are generally values assessing a subject's performance of anactivity path when compared to an idealized activity path.

Assessing Performance of a Physical Activity With a Path Tracking Model

FIG. 2 depicts an example process for developing and implementing a pathtracking model, such as 110 described above with respect to FIG. 1.

A path tracking model may be developed for use in assessing a subject'sperformance of a physical activity sequence, or a portion thereof.Development may generally begin with capturing motion data (e.g., usinga motion capture system) while a training subject (e.g., an expert)demonstrates a specific ideal activity sequence, such as at 202 and 204.Analysis of the captured motion data, including the movement of variousbody elements within an activity path between key states of the activitysequence, enables identification of primary body elements that areuseful for monitoring activity form, progression, and completion.

Recording the activity sequence (e.g., at 204) may be performed forexample by a system comprising a computer accessing a motion capturedevice and running software algorithms that process the data from thedevice. Such a system may provide motion capture data for an activity byobserving a trainer (e.g., an expert practitioner, possibly a clinicianlike a physiotherapist) enacting the activity.

A motion capture device may include optical camera systems with imageprocessing, marker-based tracking systems with various marker (active,passive, semi-passive, modulated) and detector (optical, radiofrequency) types, depth camera systems with object recognitionalgorithms, inertial measurement units, mechanical exoskeleton motioncapture systems, or magnetic flux measurement systems, to name a fewexamples.

From recorded motion capture data, the training subject's state andactivity progression may be quantified by extracting the coordinates ofthe primary joint positions of the trainer subject during the physicalactivity sequence. The training subject's body elements and limbs canthen be determined to create a skeleton reconstruction based on the datacaptured by the motion capture system. Generally, each body element maybe defined by a length between distinguishing joints and a unit vectorgiving its orientation in a given coordinate frame. A unit vector has alength of one and can be used to represent spatial direction/orientationin a given coordinate space.

Beneficially, implementing a unit vector-based approach for quantifyingactivity paths may be more robust than using Euclidean distance measuresto track subject movement. For example, calculating body element unitvectors is a normalization process that may provide better trackingcapabilities for subjects of varying biometrics.

Joint angles may also be calculated using vector operations on adjoiningbody element vectors.

Determining an Activity Path

An activity path is generally a motion path followed by a body element(e.g., a leg, an arm, a torso, etc.) while performing a physicalactivity sequence. For example, during a push-up, a subject's torso maymove through a path that goes down initially and back up again. In somecases, activity paths may be defined with reference to key states of aphysical activity sequence, such as the starting with a first key stateand ending at a second key state.

An activity path may be quantitatively defined based on identificationof one or more path characteristic metrics (e.g., 208A) thatcharacterize the activity path. In some embodiments, a pathcharacteristic metric may include a temporal aspect, such as an amountof time taken to complete the activity path.

In some embodiments, path characteristic metrics may be directly derivedfrom state characteristic metrics (e.g., 118 in FIG. 1) used in anactivity definition model (e.g., 206). For example, in one embodiment,classifiers within activity definition model 206 may be used to trackpath characteristic metrics 208A. Thus, in some embodiments, pathcharacteristic metrics may be associated with and/or defined by thestate characteristic metrics associated with the classifiers.

As another example, path characteristic metrics 208A may be directlyidentified by a clinician, doctor, physical therapist, or the like basedon observation of a physical activity sequence and professionalassessment.

In some embodiments, path characteristic metrics 208A may be based onone or more primary body elements. As above, primary body elements aregenerally components of a subject's body that undergo significant motionand/or represent the majority of the mechanical work done by a subjectin performing a physical activity sequence, or some portion thereof.

The path characteristic metrics (e.g., 208A) may then be used toquantitatively define an ideal activity path 210 as well as to trackvariations from the ideal activity path. The ideal activity path 210 fora physical activity sequence may incorporate one or more of the pathcharacteristic metrics 208A.

Determining an Ideal Activity Path

An ideal activity path (e.g., 210) may be determined and compared to asubject's actual activity path to assess the performance of the subjectin completing a physical activity sequence, or some part thereof. Insome embodiments, ideal activity path 210 includes spatial and/ortemporal information regarding how a subject should perform a physicalactivity sequence, or a portion thereof.

Ideal activity path generation may start by first collecting idealphysical activity sequence motion data, such as at 204. The capturedmotion data may be used to define ideal activity path 210, which mayinclude a quantification of the motion of one or more primary bodyelements throughout the physical activity sequence, or some portionthereof. As above, the physical activity sequence may be performed byexperts demonstrating the ideal physical activity while motion data iscaptured.

In some embodiments, an ideal activity path may be based on thecollection of several sample data sets of captured motion data fromideally performed activity sequence repetitions, such as by an expert.FIGS. 5 and 6, described in more below, demonstrates one example processdetermining an ideal activity path based on a plurality of activity pathsamples data sets.

In certain situations, it may be advantageous to capture motion datafrom multiple subjects and/or multiple tracking methodologies ordevices. Differences can be found in joint positions and body elementorientation for training subjects of varying size and shape depending onthe motion capture platform used. This provides activity paths withwider ranges of acceptable orientations and will result in more robusttracking performance for varying subjects. It should be noted that thereare situations where the requirement may be to customize the activityfor specific user(s) or devices in which case the idealized pathformulation can be restricted to those specific configurations.

Path similarity measures 212, which quantify the similarity betweenpaths, may be determined based on ideal activity path 210 and pathcharacteristic metrics 216A.

Beneficially, ideal path determination may be an automated and trainableprocess capable of using data from varying subjects. This feature mayresult in more robust and tunable tracking capabilities.

Configuration of Path Tracking Model

Tracking and assessing a subject's physical activity sequenceperformance is improved by comparing the subject's activity path 216 toideal activity path 210. In some embodiments, a path tracking model(e.g., 218) containing one or more path similarity measure (e.g., 218A),may track a subject's motion throughout an activity and may provide atracking score (e.g., 218B) for overall assessment. Tracking score 218Bgenerally provides an objective metric to assess the similarity betweenthe subject's activity path (e.g., captured in motion data at 214) andthe ideal activity path (e.g., 210).

Beneficially, tracking score outputs from a path tracking model (e.g.,218) provide users with additional information regarding theirperformance over the complete course of an activity path. This scoringinformation can be utilized as additional feedback to aid subjecttraining and may facilitate corrective actions with the intent to betterdemonstrate the proper technique desired by, for example, a prescribingclinician.

In some embodiments, the subject's activity path 216 includes pathcharacteristic metrics (e.g., 216A), which define the significantcomponents of the subject's motion throughout the course of a physicalactivity sequence, or portion thereof. Path tracking model 218 monitorsthese path characteristic metrics in order to assess a subject'sperformance of a physical activity sequence. For example, path trackingmodel 218 may compare the subject's path characteristic metrics (derivedfrom the captured motion data at 214) to ideal activity path 210 todetermine path similarity measures 218A.

In some embodiments, ideal activity path 210 is segmented based on keystates occurring during performance of the physical activity sequence.In such cases, different path characteristic metrics 216A may be usedfor each segment of the segmented ideal activity path.

In some embodiments, path similarity measures may include, for example,root mean square deviation, Minkowski distance, Frechet distance,Hausdorff distance, Mahalanobis distance, Cosine distance, radial basisfunction kernels, and others. More generally, a path similarity measurecan be any measure capable of numerically comparing the differencebetween activity paths. Advantageously, many types of path similaritymeasures may be used and configured to suit a specific end taskrequirement without any structural changes to the methods describedherein. For example, if the timing of a physical activity sequence is animportant feature, a measure with a temporal component can beimplemented to provide additional feedback.

In some embodiments, tracking score 218B may be a based on a pluralityof path similarity measures 218A. The process of combining one or morepath similarity measure 218A into a composite tracking score 218B may beconfigured, for example, to place greater emphasis on certain primarybody elements or on certain motion characteristics (temporal vs.spatial), to name a few examples.

Path tracking model 218 may be further configured to provide trackingscore feedback at various phases of a physical activity sequence, or atdifferent frequencies during a physical activity sequence. In someimplementations, continuous tracking scores may be output in real-timeduring capture of a subject's motion (e.g., at 214). In otherembodiments, tracking scores may be calculated at the end of eachphysical activity sequence, or portion thereof. For example, trackingscores may be provided at each key state in a physical activitysequence.

Use of Path Tracking Model to Monitor Subject Activity

In some embodiments, a path similarity measure (e.g., 218A) of a pathtracking model (e.g., 218) may be configured to analyze each point alongan ideal activity path (e.g., 210). Because the subject's intendedmotion may not be known a priori, path tracking model 218 may determinethe best matching ideal activity path based on the subject's performedpath. FIG. 8, described in more detail below, describes one method ofdetermining the best matching state along an ideal activity path.

As subject 212 transitions between key states of a physical activitysequence, its path characteristic metrics 216A are compared with idealactivity path 210 until the next key state in the activity sequence isachieved. During this transition between key states, the current pathcharacteristic metric values may be compared to points along an idealactivity path in order to determine a best matchingorientation/position. To this end, each point along an ideal activitypath may be considered in ideal path state represented by a statevariable set representing an ideal position and/or orientation for oneor more body elements. Thus, captured motion data may be compared toeach point along an ideal activity path (e.g., 210) so that anappropriate ideal activity path state may be determined based on subject212′s current state.

For example, when primary body element unit vectors are implemented asmetrics (as described in more detail below with respect to FIGS. 3 and4), the orientation difference between subject 212′s current state andthe best matching ideal activity path state may be expressed as anangular deviation of the primary body element(s) as compared to theideal activity path.

The angular deviation between the body element's actual and ideal unitvectors may be calculated using, for example, a vector dot-productoperation that is measured about the axis defined by the cross-productbetween the two vectors, according to:

θ=cos⁻¹({right arrow over (u)}·{right arrow over (v)})   (Eq 1)

where θ is angular deviation, {right arrow over (u)} is the actual unitvector orientation, and {right arrow over (v)} is the ideal unit vectororientation. Equation 1 thus provides the absolute angle deviationbetween an ideal body element orientation and the actual orientation ofthe body element in the captured motion data.

These orientation differences may be recorded and stored as subject 212moves between key states and may be used to calculate a spatial trackingscore for feedback. In one embodiment, the tracking score i may becalculated according to:

$\begin{matrix}{\tau = {\frac{1}{n}{\sum_{i = 1}^{n}{❘\theta_{i}❘}}}} & \left( {{Eq}.2} \right)\end{matrix}$

where θ_(i) is the angle deviation of sample i and n is the total numberof samples along the path. Equation 2 takes the average of allangle-deviation samples recorded during the activity path and is thus anaveraging equation that may be used to produce a tracking score (e.g.,218B) at the completion of an activity path.

The time durations of each key-state transition may also be compared tothat of ideal activity path 210 to produce a temporal tracking score foran activity path. The temporal tracking score may be implemented into apath similarity measure to better track activity timing characteristics.Once again, the combination of path similarity measures can beconfigured to place greater emphasis on certain characteristics of anactivity.

In some embodiments, tracking scores (e.g., 218B) may be provided to asubject (e.g., 212) in real time with little to no latency depending onthe motion capture source and supporting computer hardware. This livetracking score feedback may be used as the basis of other metrics ofinterest, like the risk of injury to subject 212 based on the subject'sdeviation from the ideal activity path 210 during an activity sequence.

For example, as subject 212 deviates further from ideal activity path210, tracking score 218B will reflect this increase. If tracking score218B deviates beyond a threshold, then a cautionary warning may beprovided to subject 212, which may beneficially result in a reduced riskof injury to subject 212.

Path tracking models may further be configured to produce a set oftracking scores, wherein each tracking score in the set of trackingscores is associated with a specific body element (e.g., an arm, a leg,etc.). Additional tracking scores may also be used to provide additionalfeedback to subject 212, such as by isolating problematic body regionsand targeting specific areas of activity performance improvement.

Note that while the aforementioned example uses primary body elementunit vectors as path characteristic metrics and segmented ideal activitypaths based on key states of a physical activity sequence, this is justone example embodiment and other embodiments may be implementeddifferently.

Example Primary Body Elements

FIG. 3 depicts an example of skeletonization output of a subject's lowerbody portion from a motion capture source with primary body elementsidentified. In particular, FIG. 3 depicts primary body elements 302 and306 in different orientations between two key states in an activitysequence. As above, primary body elements 302 and 306 may be identifiedby a clinician based on work performed by the patient during theactivity sequence or by a numerical approach.

FIG. 3 further depicts an example of primary body element unit vectors304 and 308, which may be used as path characteristic metrics (e.g.,216A in FIG. 2) in some embodiments. Notably, the methods describedherein work with any quantitative representation of an activity path. Inother words, it is not necessary that an activity path be representedonly as a set of unit vector trajectories. Further, note that it ispossible to derive a subject's joint positions using mathematicaloperations on captured motion data from tracking systems that do notdirectly provide joint positions.

Activity Paths Based on Primary Body Element Unit Vectors

FIG. 4 depicts an example of using primary body element unit vectors toderive path characteristic metrics (e.g., 208A and 216A in FIG. 2) tomeasure a subject's physical activity path.

In particular, this example depicts the differences in unit vectororientations between two key states, 402 and 404, in an activitysequence by plotting magnitude differences for each of three componentdirections (X, Y, and Z) of the primary body element unit vector. Plot406 depicts the values of the path characteristic metrics of the unitvector trajectory for a primary body element expressed as a function oftime as the subject transitions from key state 402 to key state 404.

Activity Path Normalization

FIG. 5 depicts an example of activity path normalization. In particular,FIG. 5 depicts a path completion normalization approach in which aplurality of recorded activity path samples, 502 and 504, are normalizedwith respect to time by using the start and end points of the physicalactivity sequence, which in some cases may be key states. Oncetime-normalized, the activity paths may be expressed as a percentage ofcompletion rather than in time, such as depicted in plot 506, via atransfer function, such as 508 and 510.

For example, transfer functions T₁ (508) and T₂ (510) transform thetime-domain D_(t) output of plots 502 and 504, respectively, intonormalized domain D_(p) output, as in plot 506.

In some embodiments, the normalization approach depicted in FIG. 5 maybe used for ideal activity path determination, as described above withrespect to FIG. 2.

This normalization process allows for activities of varying rates to becompared on an equal scale. In this normalized domain, collections ofnormalized paths may be combined to calculate a mean path of the samplemotion capture datasets, such as described with respect to FIG. 6.Further, a mean path trajectory may then be transformed back to the timedomain using the mean path completion time observed in the trainingdata, as shown in FIG. 7. Once transformed back into the time domain, amean path represents the idealized path for that activity.

When normalizing activity paths, all similar path characteristic metricsof the normalized activity paths may be combined using an averagingmethod and converted back into a function of time using a mean observedtime for path completion, such as depicted in FIG. 6. This processproduces a mean path as a function of the normalized time, and also astandard deviation of the path over the normalized time, as depicted inFIG. 7. A standard deviation around the mean completion time may also bedetermined.

Activity path normalization may also be done in a piecewise fashion bysegmenting and normalizing the activity path samples using each physicalactivity key state to mark the idealized path start and end point. Asabove, physical activities may be defined as a sequence of key statesthe subject must achieve over the course of a physical activity.Creating separate segments of an ideal activity path, delineated by eachkey state in the activity, may provide for a more precise normalizationprocess, and may also facilitate tracking scores with more concentrateddetails regarding certain segments of the activity path. In this case,over the course of the full activity sequence, the idealized path wouldhave a piecewise formulation for each transition region between activitykey states. Each segment of the activity path may include both thestarting and ending key-state, possibly creating some state overlap whencombined into one complete activity path.

Normalization of activity path samples may also be done using anactivity definition model (e.g., 120 in FIG. 1). Knowledge of pathcompletion percentage information may be used to improve thesynchronizing of sample activity paths before the calculation of anidealized activity path (e.g., 210 of FIG. 2). An activity definitionmodel may be used to produce a measure of path completion percentagebased on the classifier likelihoods produced comparing each transitionalstate along the path to the two key-states defining the activity pathstarting and ending points. This path completion percentage, defined asa function of time (or normalized time), allows for the creation of atransfer function T to convert the time-domain state variable data intoa normalized domain based on path completion percentage. Thisimplementation may help mitigate issues due to sample activity sequencesbeing performed at varying rates.

In some embodiments, a normalization method may be implemented usingprimary body element orientations for activity path tracking, which mayincrease robustness to differences in subject biometrics and resultingskeletonization data. In particular, activity paths may be quantifiedand normalized by tracking the primary element unit vector orientationdifferences between the key states (as in FIG. 4). For these primarybody elements, tracking is performed by analyzing the unit vectororientation changes as the subject transitions from one key state to theother.

Calculation of body element unit vectors is a normalization processbeneficially reduces the influence of variations in subject body type(height, weight, etc.). With the interest of tracking large numbers ofsubjects, with equal assessment, the use of unit vectors may be a morerobust choice for use as path characteristic metrics.

In some embodiments, a combination of path similarity measures may alsotake into account the ideal activity path standard deviation observedduring formulation of the ideal activity path.

Beneficially, a collection of activity path samples (e.g., 502 and 504)generated from captured motion data of ideally performed physicalactivity sequences may provide information regarding the acceptablelimits of deviation from an idealized. In some embodiments, a mean path(idealized path), and the standard-deviation around this mean, can becalculated and may be accounted for in the path similarity measures.

Mean Normalized Path Determination

FIG. 6 depicts an example method of utilizing averaging to combineseveral normalized activity path samples 602 into one mean normalizedpath 604. In the depicted example, each common component (e.g., statevariable) of the normalized activity path samples is averaged along thefull path according to the following equation (606 in FIG. 6):

$\begin{matrix}\left. {\frac{1}{n}{\sum P_{i}}}\rightarrow P_{l} \right. & \left( {{Eq}.3} \right)\end{matrix}$

where n is the number of normalized path samples, P_(i) is the ithnormalized path sample, and P₁ is the normalized average path. Notably,this method can be used for producing an ideal activity path, such as amean normalized ideal path.

Converting Mean Normalized Paths Into an Ideal Path

FIG. 7 depicts an example method of using a mean path completion time(as depicted in 702) to convert the idealized normalized activity pathin plot 704 into an idealized activity path expressed as a function oftime 706. In this example, transfer function T_(t) (708) may be createdusing the mean path-completion-time to transform normalized domain D_(p)output, as depicted in plots 702 and 704, into time-domain D_(t) output,as depicted in plot 706.

Ideal Activity Path State Selection

As described above, a path tracking model can contain one or more pathsimilarity measures (e.g., 218A in FIG. 2) that each produces a measureof similarity between an observed activity path (e.g., of a subjectperforming a physical activity sequence) and an ideal activity path. Theoutput of the path similarity measures can be combined to produce one ormore tracking scores providing an overall assessment of the activityperformance. The combination of the path similarity measures can beperformed in several acceptable methods and may be done to place greateremphasis on specific characteristics of the motion.

However, when tracking a subject's performance of a physical activitysequence, the path tracking model may need to determine an appropriateideal activity path state to compare with the subject's observedactivity path.

FIG. 8 depicts an example method for finding the best matching statealong an ideal activity path based on a current orientation of a primarybody element. In this example, the deviation from the currentorientation is compared to every point along the ideal activity path,and an index at which the minimum orientation difference occurs isdetermined to be the best matching state along the ideal activity path.

In particular, during the progression of an activity path, components ofa subject's observed state can be compared to an ideal activity path bya path tracking model as depicted in plot 804. Notably, FIG. 8 depictsan example of a spatial comparison independent of exercise timing orvelocity. Rather, the example method identifies the best matchingorientation 802 along the idealized path given the current orientation(in dimensions X, Y, and Z) of a primary body element. The deviation (ordifference) of the current orientation is calculated (or compared) toevery point along the idealized path, as shown in the plot 806. Theindex along the ideal activity path that shows the minimum orientationdifference is determined to be the best matching point. This point alongthe idealized path shows the closest resemblance to the current state ofthe subject.

Throughout the progression of the subject's activity path, the minimumorientation difference for each motion capture data sample isdetermined. The calculated minimum deviation (e.g., point 802) may beprovided as a tracking score for every sample of motion capture dataduring the progression of the path.

Further, once a subject completes a full motion of an activity path, andthey are in the desired final key state, the collection of orientationdifferences may be analyzed to provide an aggregate tracking score basedon the full collection of samples recorded through the entire activitypath. Thus a tracking score may be generated at the completion of everyactivity path or designated path segment.

This minimum deviation approach is one on many acceptable methods ofpath tracking using a path similarity measure. The choice of otheracceptable path similarity measures would facilitate slightly varyingprocesses. For example, using a Frechet distance metric, the subject'sobserved activity path would be compared to the idealized path after thecompletion of the activity sequence. The Frechet distance would providea measure of similarity between the paths taking into account thelocation and ordering of the samples along both paths.

Beneficially, the method of identifying a best-matching orientationalong an ideal activity path, such as depicted with respect to FIG. 8,may provide a comparison which is time invariant, which allows for amore comprehensive analysis of activity motion differences (spatial vs.temporal).

Example Activity Path Tracking User Interface

FIG. 9 depicts an example graphical user interface 900 of a liveactivity path tracking application. In this example, a subject isperforming a standing left-leg hip abduction activity and transitioningbetween the two key states of the physical activity sequence.

Path deviation tracking scores 902 are displayed at the lower edge ofthe picture, such as may be calculated by a path tracking model (e.g.,218 of FIG. 2). In this case, the tracking score (path deviation)represents the minimum angle between the current orientation of thetracked one or more body elements and that of the closest matching statealong the idealized path. Notably, each of the tracking scores in thisexample is based on a different motion capture source, such as twodifferent motion capturing camera systems. Thus, multiple motion capturesystems can be used concurrently.

Beneficially, the tracking score (e.g., 902) provided by the pathtracking model may be used to provide a prediction of risk and allow forcautionary feedback to prevent potential injury during an activity. Ifthe subject deviates greatly from the ideal path, live feedback from thepath tracking model may reduce the risk of potential injury.

Alternative Approach Using Vector Algebra

FIG. 10 provides an example of an alternative approach for plotting andcomparing activity paths using vector algebra.

Initially, each element of an activity path (e.g., 208 or 216 of FIG. 2)may be considered to be a vector of values each of which relates to oris derived from a set of state variables (for example, a vector of thepath characteristic metrics). Each vector element of the activity pathmay be augmented with one or more time-based variables, which depict thetemporal aspect of the activity. For example, a time augmentation couldbe the real time elapsed (in suitable units) since the beginning of theactivity, or it could be the percentage of time elapsed relative to thetime for the entire activity (or the activity between key states). Anenhanced activity path may now be represented by a sequence of theseaugmented vectors, ordered temporally or otherwise.

Mathematically, the sequence of these augmented vectors defines a pathin a space with an appropriate number of dimensions, this path havingthe same number of dimensions as the augmented vector, such as shown inFIGS. 10A and 10B from different perspectives. To compute the distancebetween paths in this space, for example an ideal activity path 1002 andan activity path followed by a specific subject 1004, a distancefunction (which is a form of path similarity measure) can be defined onthis space based on various approaches, such as a Minkowskidistance-based metric, a Hausdorff distance, a Frechet distance, andothers. Note that individual variables in the vectors may also be scaledrelative to the rest, prior to the application of the distance function.

For example, time derived variables can be scaled higher or lowerdepending on whether the time aspect is more critical or less criticalthan the spatial aspects of the activity. The same effect can also beachieved by appropriately defining the distance function rather thanscaling the elements of the augmented vector.

Example Method of Assessing Performance of a Physical Activity

FIG. 11 depicts an example method 1100 of assessing the performance of aphysical activity.

Method 1100 begins at step 1102 with recording motion capture data whilea training subject demonstrates a physical activity sequence, such asdescribed above with respect to FIG. 2.

Method 1100 then proceeds to step 1104 with identifying one or moreprimary body elements based on the motion capture data, such asdescribed above with respect to FIG. 3.

Method 1100 then proceeds to step 1106 with deriving one or more pathcharacteristic metrics based on a state variable set and the one or moreprimary body elements, wherein the state variable set defines the stateof a body at any given time, such as described above with respect toFIG. 2.

Method 1100 then proceeds to step 1108 with defining an ideal activitypath of the physical activity sequence based on the one or more pathcharacteristic metrics, such as described above with respect to FIGS. 2and 7.

Method 1100 then proceeds to step 1110 with tracking a subject's motionthroughout a performance of the physical activity sequence using a pathtracking model, wherein the path tracking model comprises one or morepath similarity measures, such as described above with respect to FIG.2.

Method 1100 then proceeds to step 1112 with generating one or moretracking scores based on the one or more path similarity measures, suchas described above with respect to FIGS. 2 and 9.

In some embodiments, method 1100 further includes determining thesimilarity of the performance of the physical activity sequence to thatof the ideal path for the physical activity sequence based on thetracking scores.

In some embodiments, method 1100 further includes recommendingcorrective actions to the subject based on the tracking scores.

In some embodiments of method 1100, the primary body elements correspondto limbs of the trainer.

In some embodiments of method 1100, identifying the primary bodyelements includes: quantifying work performed by a plurality of bodyelements in the motion capture data; and selecting as primary bodyelements a subset of the plurality of body elements performing workabove a work threshold.

In some embodiments of method 1100, the ideal activity path is furtherdefined based on multiple recorded physical activity sequence samplescaptured in the motion capture data, such as described above withrespect to FIGS. 5-7.

In some embodiments of method 1100, all recorded physical activitysequence samples are normalized with respect to a representation of pathcompletion of the physical activity sequence, such as described abovewith respect to FIG. 5.

In some embodiments of method 1100, the one or more path characteristicmetrics comprise primary body element unit vectors, such as describedabove with respect to FIGS. 3 and 4.

In some embodiments of method 1100, at least one of the one or more pathsimilarity measures comprises a measure of similarity between curvesthat takes into account the location and ordering of the points alongthe curves, for example, like the Frechet distance.

Notably, method 1100 is just one example, and many others are possibleas described herein.

Example Processing System

FIG. 12 depicts an example processing system 1200 configured to generateand use activity path tracking models.

For example, processing system 1200 may be configured to perform one ormore aspects of the flows described with respect to FIG. 2 and method1100 described with respect to FIG. 11.

Processing system 1200 includes a CPU 1202 connected to a data bus 1250.CPU 1202 is configured to process computer-executable instructions,e.g., stored in memory 1210, and to cause processing system 1200 toperform methods as described herein. CPU 1202 is included to berepresentative of a single CPU, multiple CPUs, a single CPU havingmultiple processing cores, and other forms of processing architecturecapable of executing computer-executable instructions.

Processing system 1200 further includes input/output device(s) 1204,which may include motion capture or tracking devices as describedherein, as well as input/output interface(s) 1206, which allowprocessing system 1200 to interface with input/output devices, such as,for example, keyboards, displays, mouse devices, pen input, motioncapture or tracking devices, motion tracking sensors, and other devicesthat allow for interaction with processing system 1200.

Processing system 1200 further includes network interface 1208, whichprovides processing system 1200 with access to external networks, suchas network 1214.

Processing system 1200 further includes memory 1210, which in thisexample includes a plurality of components.

For example, memory 1210 includes motion capture component 1212, bodyelement identification component 1214, path characteristic metriccomponent 1216, ideal activity path component 1218, subject trackingcomponent 1220, scoring component 1222, and recommendation component1224, each of which may be configured to perform various aspects of themethods described herein, including method 1100 described with respectto FIG. 11.

Memory 1210 further includes state variables 1232, state characteristicmetrics 1234, activity definition models 1236, activity paths 1238, pathcharacteristic metrics 1240, path similarity measures 1242, and pathtracking models 1244, each of which may be configured to support variousaspects of the methods described herein, including method 1100 describedwith respect to FIG. 11.

Note that FIG. 12 depicts various example aspects stored in memory 1210,but others are possible consistent with the systems and methodsdescribed herein. Further, while shown as a single memory 1210 in FIG.12 for simplicity, the various components stored in memory 1210 may bestored in different memories, but all accessible CPU 1202 via internaldata connections, such as bus 1250, and external data connections, suchas network interface 1208.

Notably, while shown as a single processing system in the exampledepicted in FIG. 12, other embodiments may include distributed processesthat function together as a processing system. For example, the variousaspects in memory 1210 may be implemented or stored across a network ofprocessing systems, or in a cloud-based processing system, or incombinations of the same.

For example, a patient may have a client processing system that includesa motion tracking I/O device that captures lives data and feeds it backto a server processing system. Similarly, the patient's clientprocessing system may store path tracking models 1244 locally, whichwere generated remotely, and which were downloaded to the local clientprocessing system over a network connection, such as the Internet.

Further, processing system 1200 may be configured to function as atraining system or a tracking system. Other embodiments of processingsystems may be a training system only, or a tracking system only. Forexample, patients may receive only tracking systems.

In general, processing system 1200 is just one possible embodiment, andthe various aspects of processing system 1200 may be distributed acrossa plurality of devices, may be omitted, or added as necessary for any ofthe particular functions or methods described herein.

Alternative Implementations and Other Capabilities

Though a few example implementations are described herein, there arevarious ways of implementing the methods described herein.

Path tracking models (e.g., 218 in FIG. 2) may also be used to detecterroneous motions one wishes to avoid during a physical activitysequence. To this end, an activity path representing anerroneous/non-ideal activity sequence can be calculated using data fromrecorded non-ideal activity motion capture data samples. Then a pathtracking model can be formulated to identify the equivalence of thesubject's motions to this non-ideal activity path. The resultingtracking score can be used to detect when the subject is performing theactivity in this non-ideal manner. These erroneous/non-ideal motions mayinclude motions that could potentially result in injury or are engagingincorrect body regions.

Patient screening may be performed with the help of a path trackingmodel. Patients with issues of limited range of motion or inflexibilityof certain joints associated with common ailments, may performactivities in a distinguishable manner. Specific idealized paths, usedfor screening, can be formulated based on motion capture data samplesfrom subjects of a common condition to detect this specific condition.When live tracking, the subject's similarity to this screening idealizedpath can be calculated to assess the potential for the subject topossess the condition of interest.

Activity sequence progression can also be calculated using a process ofmatching the subject's current state to the best matching state alongthe idealized path. The index of the best matching state along theidealized path can be used to estimate the percentage of completion ofthe associated activity path. This may be used to provide the subjectwith live feedback of their progression performing a prescribedactivity.

The methods described herein may further be configured to trackcompliance and potential activity form degradation throughout the courseof performing a set of prescribed activity sequences. Tracking scorevalues may be used to determine if the patient is truly demonstratingthe correct motion required for the activity during each repetition inthe set. Tracking score value variations over the course of a set ofactivity sequence repetitions may be monitored and ultimately used tomake alterations to the prescribed activity.

Formulation of path tracking models and ideal activity paths maygenerally require quantified pose information. This data may come fromvarying sources or methods other than a motion capture source aspreviously described. For example, an alternative motion capture devicemay include optical camera systems with image processing, marker basedtracking systems with various marker (active, passive, semi-passive,modulated) and detector (optical, radio frequency) types, depth camerasystems with object recognition algorithms, inertial measurement units,mechanical exoskeleton motion capture systems, or magnetic fluxmeasurement systems. Other potential sources include, but are notlimited to, data extracted from inertia-measurement-units (IMUs) andimage processing methods that compare sequential images to determinedifferences. Depth cameras and/or point cloud mapping can also be usedto extract information on varying states.

Path tracking models can also be formulated from data expressed invarious other coordinate spaces rather than those discussed herein. Forexample, two-dimensional (2D) data could be extracted from imageprocessing techniques, or 3D motion capture devices may project trackingdata onto a 2D plane.

Path tracking models may be time-invariant, as described above, but arenevertheless fully capable of tracking and comparing activity sequencetiming. Ideal timing can be extracted from the recorded motion capturedata to determine the desired rate of transition between key states.These transition periods between key states can be identified usingclassifiers within an activity definition model. With discreteboundaries established between key states and transition zones, timeinformation can be recorded and binned into associated keystate/transition regions. Subject timing during key state transitionscan be compared to that from the idealized activity motion recordedduring path tracking model formulation.

Various statistical and comparative methods can be applied for pathsimilarity assessment and implemented for use as a path similaritymeasures. These include, but are not limited to, root mean squaredeviation, Minkowski distance based functions, Frechet distance,Hausdorff distance, signal cross-correlation, cross-covariance, dynamictime warping, and others.

For idealized path formulation, the activity sequence can be segmentedinto sections between each key state in the activity (as describedabove) to create a piecewise idealized path for the activity sequence.An alternative implementation is also possible where a continuousidealized path can be calculated for the full activity sequence.

Use of path tracking models for physical activity tracking has beendescribed herein for rehabilitation applications and can be applied tomany other tasks. This includes biomechanical analysis of physicalactivity training and sport science research, such as training properform or detection of critical movements. This method also has potentialapplications in clinical sciences such as the analysis of posture,balance, gait, and motor control. The same approach can be used forgesture/pose recognition and detection in virtual reality, gamingapplications, robotics, manufacturing applications, and ergonomicstudies. Path comparative models can also be applied in psychologicalstudies for analysis on behavioral and physical response.

Example Clauses

Clause 1: A method for assessing the performance of a physical activity,comprising: recording motion capture data while a training subjectdemonstrates a physical activity sequence; identifying one or moreprimary body elements based on the motion capture data; deriving one ormore path characteristic metrics based on a state variable set and theone or more primary body elements, wherein the state variable setdefines the state of a body at any given time; and defining an idealactivity path of the physical activity sequence based on the one or morepath characteristic metrics.

Clause 2: The method of Clause 1, further comprising: tracking asubject's motion throughout a performance of the physical activitysequence using a path tracking model, wherein the path tracking modelcomprises one or more path similarity measures; and generating one ormore tracking scores based on the one or more path similarity measures.

Clause 3: The method of Clause 2, further comprising: determining thesimilarity of the performance of the physical activity sequence to thatof the ideal activity path for the physical activity sequence based onthe tracking scores.

Clause 4: The method of Clause 3, further comprising: recommendingcorrective actions to the subject based on the tracking scores.

Clause 5: The method of any one of Clauses 1-4, wherein the one or moreprimary body elements correspond to limbs of the training subject.

Clause 6: The method of any one of Clauses 1-5, wherein identifying theone or more primary body elements comprises: quantifying work performedby a plurality of body elements in the motion capture data; andselecting as the one or more primary body elements a subset of theplurality of body elements performing work above a work threshold.

Clause 7: The method of any one of Clauses 1-6, wherein the idealactivity path is further defined based on multiple recorded physicalactivity sequence samples captured in the motion capture data.

Clause 8: The method of Clause 7, wherein all recorded physical activitysequence samples are normalized with respect to a representation of pathcompletion of the physical activity sequence.

Clause 9: The method of any one of Clauses 1-8, wherein the one or morepath characteristic metrics comprise primary body element unit vectors.

Clause 10: The method of Clause 2, wherein at least one of the one ormore path similarity measures comprises a measure of similarity betweencurves that takes into account the location and ordering of the pointsalong the curves.

Clause 11: A processing system, comprising: a memory comprisingcomputer-executable instructions; and one or more processors configuredto execute the computer-executable instructions and cause the processingsystem to perform a method in accordance with any one of Clauses 1-10.

Clause 12: A non-transitory computer-readable medium comprisingcomputer-executable instructions that, when executed by one or moreprocessors of a processing system, cause the processing system toperform a method in accordance with any one of Clauses 1-10.

Clause 13: A computer program product embodied on a computer readablestorage medium comprising code for performing a method in accordancewith any one of Clauses 1-10.

Additional Considerations

The preceding description is provided to enable any person skilled inthe art to practice the various embodiments described herein. Theexamples discussed herein are not limiting of the scope, applicability,or embodiments set forth in the claims. Various modifications to theseembodiments will be readily apparent to those skilled in the art, andthe generic principles defined herein may be applied to otherembodiments. For example, changes may be made in the function andarrangement of elements discussed without departing from the scope ofthe disclosure. Various examples may omit, substitute, or add variousprocedures or components as appropriate. For instance, the methodsdescribed may be performed in an order different from that described,and various steps may be added, omitted, or combined. Also, featuresdescribed with respect to some examples may be combined in some otherexamples. For example, an apparatus may be implemented or a method maybe practiced using any number of the aspects set forth herein. Inaddition, the scope of the disclosure is intended to cover such anapparatus or method that is practiced using other structure,functionality, or structure and functionality in addition to, or otherthan, the various aspects of the disclosure set forth herein. It shouldbe understood that any aspect of the disclosure disclosed herein may beembodied by one or more elements of a claim.

As used herein, the word “exemplary” means “serving as an example,instance, or illustration.” Any aspect described herein as “exemplary”is not necessarily to be construed as preferred or advantageous overother aspects.

As used herein, a phrase referring to “at least one of” a list of itemsrefers to any combination of those items, including single members. Asan example, “at least one of: a, b, or c” is intended to cover a, b, c,a-b, a-c, b-c, and a-b-c, as well as any combination with multiples ofthe same element (e.g., a-a, a-a-a, a-a-b, a-a-c, a-b-b, a-c-c, b-b,b-b-b, b-b-c, c-c, and c-c-c or any other ordering of a, b, and c).

As used herein, the term “determining” encompasses a wide variety ofactions. For example, “determining” may include calculating, computing,processing, deriving, investigating, looking up (e.g., looking up in atable, a database or another data structure), ascertaining and the like.Also, “determining” may include receiving (e.g., receiving information),accessing (e.g., accessing data in a memory) and the like. Also,“determining” may include resolving, selecting, choosing, establishingand the like.

The methods disclosed herein comprise one or more steps or actions forachieving the methods. The method steps and/or actions may beinterchanged with one another without departing from the scope of theclaims. In other words, unless a specific order of steps or actions isspecified, the order and/or use of specific steps and/or actions may bemodified without departing from the scope of the claims. Further, thevarious operations of methods described above may be performed by anysuitable means capable of performing the corresponding functions. Themeans may include various hardware and/or software component(s) and/ormodule(s), including, but not limited to a circuit, an applicationspecific integrated circuit (ASIC), or processor. Generally, where thereare operations illustrated in figures, those operations may havecorresponding counterpart means-plus-function components with similarnumbering.

The following claims are not intended to be limited to the embodimentsshown herein, but are to be accorded the full scope consistent with thelanguage of the claims. Within a claim, reference to an element in thesingular is not intended to mean “one and only one” unless specificallyso stated, but rather “one or more.” Unless specifically statedotherwise, the term “some” refers to one or more. No claim element is tobe construed under the provisions of 35 U.S.C. § 112(f) unless theelement is expressly recited using the phrase “means for” or, in thecase of a method claim, the element is recited using the phrase “stepfor.” All structural and functional equivalents to the elements of thevarious aspects described throughout this disclosure that are known orlater come to be known to those of ordinary skill in the art areexpressly incorporated herein by reference and are intended to beencompassed by the claims. Moreover, nothing disclosed herein isintended to be dedicated to the public regardless of whether suchdisclosure is explicitly recited in the claims.

We claim:
 1. An apparatus comprising: at least one processor; a memorycoupled to the at least one processor, the memory comprisinginstructions that, when executed by the at least one processor, causethe at least one processor to: access training motion capture data of abody of a training subject performing a physical activity sequencecomprising a sequence of states, determine at least one primary bodyelement for the physical activity sequence, and determine a pathtracking model configured to numerically represent a path of the atleast one primary body element as the body performs the physicalactivity sequence, the path tracking model determined via: determiningat least one path characteristic metric of the at least one primary bodyelement during each of the sequence of states, the at least one pathcharacteristic metric comprising a numeric variable defining stateinformation of the at least one primary body element during one of thesequence of states, and determining an ideal activity path of thephysical activity sequence, the ideal activity path comprising anordered sequence of the at least one path characteristic metric duringthe sequence of states.
 2. The apparatus of claim 1, the stateinformation of the at least one primary body element comprising at leastone of a position or an orientation of the at least one primary bodyelement.
 3. The apparatus of claim 1, the instructions, when executed bythe at least one processor, to cause the at least one processor to:quantify work performed by a plurality of body elements in the trainingmotion capture data, and select at least one of the plurality of bodyelements performing work above a threshold as the at least one primarybody element.
 4. The apparatus of claim 1, the instructions, whenexecuted by the at least one processor, to cause the at least oneprocessor to determine a plurality of key states of the physicalactivity sequence based on the training motion capture data, each of theplurality of key states comprising a specific state that in-part definesthe physical activity sequence.
 5. The apparatus of claim 4, the idealactivity path comprising a sequence of the plurality of key states. 6.The apparatus of claim 1, the path tracking model comprising at leastone path similarity measure numerically quantifying a similarity betweenan actual activity path and the ideal activity path.
 7. The apparatus ofclaim 1, the at least one path characteristic metric comprising a unitvector of the primary body element.
 8. The apparatus of claim 1, the atleast one path characteristic metric associated with temporalinformation indicating an amount of time to complete at least a portionof the ideal activity path.
 9. The apparatus of claim 1, the idealactivity path comprising a multi-dimensional path of the at least oneprimary body element between each of the sequence of states, eachdimension of the multi-dimensional path comprising a spatial dimensionassociated with the at least one primary body element.
 10. The apparatusof claim 1, the ideal activity path further defined based on multiplerecorded physical activity sequence samples captured in the trainingmotion capture data.
 11. A computer-implemented method, comprising, viaat least one processor of a computing device: accessing training motioncapture data of a body of a training subject performing a physicalactivity sequence comprising a sequence of states; determining at leastone primary body element for the physical activity sequence; anddetermining a path tracking model configured to numerically represent apath of the at least one primary body element as the body performs thephysical activity sequence, the path tracking model determined via:determining at least one path characteristic metric of the at least oneprimary body element during each of the sequence of states, the at leastone path characteristic metric comprising a numeric variable definingstate information of the at least one primary body element during one ofthe sequence of states, and determining an ideal activity path of thephysical activity sequence, the ideal activity path comprising anordered sequence of the at least one path characteristic metric duringthe sequence of states.
 12. The method of claim 11, the stateinformation of the at least one primary body element comprising at leastone of a position or an orientation of the at least one primary bodyelement.
 13. The method of claim 11, comprising: quantifying workperformed by a plurality of body elements in the training motion capturedata, and selecting at least one of the plurality of body elementsperforming work above a threshold as the at least one primary bodyelement.
 14. The method of claim 11, comprising determining a pluralityof key states of the physical activity sequence based on the trainingmotion capture data, each of the plurality of key states comprising aspecific state that in-part defines the physical activity sequence. 15.The method of claim 14, the ideal activity path comprising a sequence ofthe plurality of key states.
 16. The method of claim 11, the pathtracking model comprising at least one path similarity measurenumerically quantifying a similarity between an actual activity path andthe ideal activity path.
 17. The method of claim 11, the at least onepath characteristic metric comprising a unit vector of the primary bodyelement.
 18. The method of claim 11, the at least one pathcharacteristic metric associated with temporal information indicating anamount of time to complete at least a portion of the ideal activitypath.
 19. The method of claim 11, the ideal activity path comprising amulti-dimensional path of the at least one primary body element betweeneach of the sequence of states, each dimension of the multi-dimensionalpath comprising a spatial dimension associated with the at least oneprimary body element.
 20. The method of claim 11, the ideal activitypath further defined based on multiple recorded physical activitysequence samples captured in the training motion capture data.